Wittenberg University - Master of Science in Analytics

ANLT 510 - Advanced Statistics and Modeling

The landscape of machine Learning Models

28 Sep 2021

Overview

Supervised Vs Unsupervised Learning Algorithms

Classes of Machine learning algorithms with applications (reference: https://medium.com/@sanchittanwar75/introduction-to-machine-learning-and-deep-learning-bd25b792e488)

Classes of Machine learning algorithms with applications (reference: https://medium.com/@sanchittanwar75/introduction-to-machine-learning-and-deep-learning-bd25b792e488)

Supervised learning algorithms can be divided into two main classes

Regression algorithms: when the output/response variable is numeric and continuous

Classification algorithms: when the output/response variable is discrete or categorical

Model Complexity vs. Model Accuracy

The big picture of supervised learning algorithms

Common elements of supervised learning algorithms

Supervised learning Example

Elements of supervised learning algorithms: #1 Data

Plot of some ideal data

Plot of some ideal data

Elements of supervised learning algorithms: #2 an assumed form of \(f_{\text{imperfect}}\)

Fitting the ideal data with a "perfect" model

Fitting the ideal data with a “perfect” model

Elements of supervised learning algorithms: #3 parameter values

Elements of supervised learning algorithms: #4 loss functions

Supervised Learning Example (Cont.)

Visualizing loss functions

A Naive loss function

\[ Loss_{_{naive}}(\mathbf{y},\mathbf{x},m,b) = \sum_{i=1}^N y_i-m\times x_i-b. \]

loss_naive <- function(params,x,y) {

  if(length(params) != 2) stop("Params should be a length 2 vector")
  
  m <- params[1]
  b <- params[2]
  
  return(sum(y - m * x - b))

}
optim(par = c(1,1),
      fn = loss_naive,
      x = df$x,
      y = df$y,
      control = list(fnscale = 1))
$par
[1] 2.684172e+55 4.996335e+54

$value
[1] -1.111779e+57

$counts
function gradient 
     501       NA 

$convergence
[1] 1

$message
NULL



A good loss function

\[ Loss_{_{absolute}}(\mathbf{y},\mathbf{x},m,b) = \sum_{i=1}^N \Big\vert y_i-m\times x_i-b\Big\vert. \]

# First define a function to optimize
loss_absolute <- function(params,x,y) {

   if(length(params) != 2) stop("Params should be a length 2 vector")
  
   m <- params[1]
   b <- params[2]
   
   return(sum(abs(y - m * x - b)))

}
optim(par = c(1,1),           # provide starting values for m and b
      fn = loss_absolute,     # define function to optimize
      x = df$x,               # provide values for known parameters
      y = df$y,               # provide values for known parameters
      control = list(fnscale = 1))
$par
[1] 5 3

$value
[1] 3.301596e-06

$counts
function gradient 
     121       NA 

$convergence
[1] 0

$message
NULL



A better loss function

\[ Loss_{_{convex}}(\mathbf{y},\mathbf{x},m,b) = \sum_{i=1}^N \Big( y_i-m\times x_i-b\Big)^2. \]

loss_convex <- function(params,x,y) {

  if(length(params) != 2) stop("Params should be a length 2 vector")
  
  m <- params[1]
  b <- params[2]
  
  return(sum((y - m * x - b) ^ 2))    

}
optim(par = c(1,1),        # provide starting values for m and b
      fn = loss_convex,    # define function to optimize
      x = df$x,            # provide values for known parameters
      y = df$y,            # provide values for known parameters
      control = list(fnscale = 1))
$par
[1] 4.999800 3.000373

$value
[1] 2.501828e-06

$counts
function gradient 
      71       NA 

$convergence
[1] 0

$message
NULL



Summary

In this presentation…

Questions

knitr::include_graphics(find_resource('images', "questions.gif"))